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Abstract 

Sequential Compressive Sensing, which may be widely used in sensing devices, is a 
popular topic of recent research. This paper proposes an online recovery algorithm for 
sparse approximation of sequential compressive sensing. Several techniques including 
• warm start, fast iteration, and variable step size are adopted in the proposed algorithm 

to improve its online performance. Finally, numerical simulations demonstrate its better 
performance than the relative art. 
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Compressive sensing (CS) [1, 2] is a recently proposed concept that enables sampling 
below Nyquist rate, without (or with little) sacrificing reconstruction quality. Based on 
exploiting the signal sparsity in typical domains, CS methods can be used in the sensing 
devices, such as MR imaging [3] and AD conversion [4], where the devices have a high cost of 
acquiring each additional sample or a high requirement on time. Therefore, as the sparsity 
level is often not known a priori, it can be very challenging to use CS in practical sensing 
hardware. 

Sequential compressive sensing [5] can effectively deal with the above problems. Se- 
quential CS considers a scenario where the observations can be obtained in sequence, and 
computations with observations are performed to decide whether these samples are enough. 
Consequently, it is allowed to recover the signal either exactly or to a given tolerance from 
the smallest possible number of observations. There have been several recovery algorithms 
for sequential CS. Asif [6] solved the problem by homotopy method. Garrigues [7] discussed 
the Lasso problem with sequential observations. 
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This work extends a recent proposed zero-point attracting projection (ZAP) algorithm 
[8] to the scenario of sequential CS. ZAP employs an approximate Iq norm as the sparsity 
constraint and updates in the solution space. Comparing with the existing algorithms, 
it needs fewer measurements and lower computation complexity. Therefore the new algo- 
rithm can provide a much more appropriate solution for practical sensing devices, which is 
validated by numerical simulations. 

2 Background 

2.1 Compressive sensing 

Suppose x is an unknown sparse signal, which is A^-length but has only K nonzero 
entries, where K <C TV. In their ice-breaking contributions, Candes et al suggested to 
measure x with under-determined observations, i.e. y = Ax, where A consists of random 
entries and has much fewer rows than columns. They also proved that Iq norm or l\ norm 
constraint optimization can successfully recover the unknown signal with overwhelming 
probability, 

x = argmin ||x|| ori) subject to y = Ax. (1) 

X 

There are many methods proposed to solve (1), of which concerned in this work is ZAP. 

2.2 Zero-point attracting projection 

ZAP iteratively searches the sparsest result in solution space. The recursion starts 
from the least square optimal solution, x(0) = Aty, where At = A T (AA T ) _1 denotes the 
Pseudo-inverse matrix of A. In the nth iteration, the solution is first updated along the 
negative gradient direction of a sparse penalty, 

x(n + 1) = x(n) - k ■ Vg(x(n)), (2) 

where </(■) denotes a sparse constraint function and k denotes the step size. In the reference, 
an approximate Iq norm is employed and the corresponding ith entry of Vg(-) is 

m lM / « • sgn(xj) - a 2 Xi \xi\ < s ; 

| elsewhere, 

where a is a controlling parameter and it is readily recognized that the penalty tends to lo 
norm as a approaches to infinity. Then x(n + 1) is projected back to the solution space to 
satisfy the observation constraint, 

x(n + l) =Px(n + l)+Q, (4) 

where P = I — A^A is defined as projection matrix and Q = A^y. Equation (2) appears 
that an attractor locates at the zero-point is pulling the iterative solution to be sparser, as 
explains the first part of the algorithm's name. The last part comes from (4), which means 
that x(n + 1) is projected back to the solution space. 
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2.3 Sequential compressive sensing 



Imagining a scenario that the samples are measured in realtime. At time m, an m-length 
measurement vector y m = A m x is collected and utilized to solve the sparsest solution by 
(1). If the available measurements are not enough to recover the original sparse signal, a 
new sample y m +i = a m+i x * s generated at time m + 1, where a m+ i denotes the sampling 
weight vector. Thus the problem becomes solving y m +i = A m+ ix, where 



Vm+l 



L a m+1 



Obviously, it is a waste of resources if the recovery algorithm is re-initialized without the 
utilization of earlier estimate, i.e. the available result at time m. Consequently, the basic 
aim of sequential compressive signal reconstruction is to find an effective method of refining 
x m+ i based on the information of x m . 



3 Online ZAP for sequential compressive sensing 

For conciseness, the detailed iteration procedure of online ZAP is provided in Tab.l. 
It can be seen that online ZAP has two recursions. The inner iteration is to update the 
solution by ZAP with the given measurements. The outer iteration is for sequential input. 
In order to improve the performance, several techniques are used in online ZAP and they 
are discussed in the following subsections. 



Table 1: The Procedure of Online ZAP 

Input: a,Ko,r]i,r]2,T,Q; 

Initialize online ZAP: xo(0) = 0,m = 0,rto = 0. 
Repeat: (for time instant m); 

for the new a m+ i and y m +i, calculate T m+ i by (5) 
and then produce P m +i and Q m +i; 

decrease step size by n m+ i = r]\K m ; 

m = m + 1; 

Initialize ZAP: n = 0, x TO (0) = x TO _i(n TO _i), k = K m ; 
Repeat: (for the nth iteration of ZAP); 

Update x m (n + 1) with the zero attraction by (2) and (3); 

Project x m (n + 1) back to the solution space by (4); 

if ||x m (n + l)||i > ||x m (n)||i and k > K m /Q, 
decrease the step size by n = rj 2 K; 

n = n + 1; 
Until: n > T or n < n m /Q. 
n m =n-l; 
Until: online ZAP stop criterion is satisfied. 
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3.1 Warm Start 



ZAP works in an iterative way to produce a sparse solution via recursion in the solution 
space. In the online scenario, the previous estimate can be used to initialize the incoming 
iteration, i.e. x m+ i(0) = x m (n m ), where n m denotes the maximum iteration number at 
time m. 



3.2 Fast Iteration 

The Pseudo- inverse matrix At plays an important role in the recursion of (4). Con- 
sidering the high computational cost of matrix inverse operation, At is generally pre- 
pared before iterations. However, in the online scenario, A m becomes time-dependent and 
Am = A m (A m A m ) 1 need to be recalculated in each time instant. In order to reduce the 
complexity, the Pseudo-inverse matrix is updated iteratively. 

Define Y m = (A m A m ) \ which is already available after time m. Consequently, as the 
new sample is arriving, using basic algebra one has the recursion 



■ m+l 



where 



L m a m+1) 



T 

a m+l_ 



[A m a m+ i] 



-1 
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m 
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a m+l a "t+l) 



(5) 
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3.3 Variable step size 

As the step size in gradient descent iterations, the parameter k controls a tradeoff 
between the speed of convergence and the accuracy of the solution. In order to improve 
the performances of the proposed algorithm, the idea of variable step size is taken into 
consideration. The control scheme is rather direct: k is initialized to be a large value after 
new sample arrived, and reduced by a factor as long as the iteration is convergent. The 
reduction is repeated several times until k is sufficiently small. Since the algorithm has two 
recursions, we employ r/i and 772 to denote the decreasing speed of outer and inner iteration, 
respectively. In addition, k is no longer decreased when the step size is rather small. 



3.4 Stop Rules 

There are two kinds of recursions requiring stop rules in the online ZAP algorithm. 
Firstly, after the mth sample arrived, x m (n) iterates with n to produce the best estimate 
based on the m measurements. The inner iteration should stop after the algorithm reaches 
steady state, which means the sparsity penalty starts increasing. Consequently, the inner 
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Figure 1: The reconstruction MSD versus the number of measurements M. 

recursion stops (a) when the number of reductions of k reaches one-Qth of its initial value 
or (b) when the number of iterations reaches the bound T. 

Secondly, as soon as the sparse signal is successfully reconstructed, the following samples 
are no longer necessary and the sensing procedure stops. Therefore, the outer recursion stops 
when the estimate error is below a particular value e. 

4 Experiment and Discussion 

Computer simulation is presented in this section to verify the performance of the pro- 
posed algorithm compared with typical sequential CS reconstruction algorithm for solving 
BPDN problem [6], whose MATLAB code can be downloaded from the website [9]. In the 
following experiment, the entries of each row of A are independently generated from normal 
distribution. The locations of K nonzero coefficients of sparse signal x are randomly chosen 
with uniform distribution [l,iV]. The corresponding nonzero coefficients are Gaussian with 
mean zero and unit variance. The system parameters are N = 256 and K = 20. The num- 
ber of measurements M increases form 1 to 120. The parameters for BPDN are set as the 
recommended values by the author. The parameters for online ZAP are a = 1, kq = 0.02, 
rji = 0.99, r/2 = 0.8, T = 50, Q = 2000. The simulation is repeated ten times, then Mean 
Square Derivation (MSD) between the original signal and reconstruction signal as well as 
the average running time calculated. 

Figure 1 shows MSD curve according to M. As can be seen, the performance of ZAP is 
better than that of BPDN. When the sparse signal is recovered successfully, the number of 
measurements BPDN needs is larger than 80, while the number ZAP algorithm needs is less 
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Figure 2: The CPU running time versus the number of measurements M . 

than 80. Figure 2 demonstrates the CPU running time as M increases. Again, ZAP has 
the better performance. The CPU time of BPDN is twice than that of ZAP for successful 
recovery (according to Fig.l, here M is chosen as 80 for comparison). 

5 Conclusion 

We have introduced in this paper a new online signal reconstruction algorithm for se- 
quential compressive sensing. The proposed algorithm extends ZAP to sequential scenario. 
And in order to improve the performance, some methods, including the warm start and 
variable step size, are adopted. The final experiment indicates that the proposed algorithm 
needs less measurements and less CPU time than the reference algorithm. 
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